Nonlinear Programming Overview

Principles

Definition

Nonlinear programming is a subfield of mathematical optimization concerned with the theory and methods for solving problems in which the objective function or any of the constraints are nonlinear functions of the variables. This area investigates techniques to find optimal solutions for complex systems where relationships are not strictly proportional, providing essential tools for modeling and decision-making across numerous scientific, engineering, and economic domains.

Ontological type

Core Methods

Problem Classes

Applications

Key developments

Key Figures

Penalty and Subproblem Methods era

The Penalty and Subproblem Methods era (1965–1987) saw constraint handling through penalty-based reformulations and master-subproblem decomposition that framed constrained NLP as tractable sequences of subproblems. Fiacco and Ishizuka developed sequential unconstrained minimization techniques and penalty-function frameworks that turn constrained problems into successive unconstrained subproblems with increasing penalty parameters. Rockafellar's Banach-space Lagrange multiplier theory provided a rigorous nonsmooth-optimization foundation for these approaches, while J. F. Benders popularized master-subproblem architectures that separate a difficult problem into solvable subproblems. In the later phase, Grossmann and Duran introduced the outer-approximation framework for MINLP, and Katarina Svanberg's moving-asymptotes method offered stabilized, convex subproblems that bridged penalty ideas with proximal-type strategies.

Global Convergence and Scalability era

Lorenz T. Biegler [1] is a leading figure in the Global Convergence and Scalability era, with affiliations at Carnegie Mellon University [3] and Aalto University [4] during 1988–2008. His key contribution in this era centers on the interior-point filter line-search method for large-scale nonlinear programming, as demonstrated by the paper On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming [5], which boosted convergence guarantees and scalability. Andreas Wächter [2] is another influential figure in this era who has been associated with Carnegie Mellon University [3]. Wächter's work, including co-authorship of the interior-point filter line-search paper [5], helped develop practical, scalable interior-point strategies for large-scale nonlinear programs, aligning with the era's emphasis on robust convergence and computational efficiency.

Hybrid and Symbolic-Numeric Integration era

Moritz Diehl [1] is a leading researcher in nonlinear programming in the Hybrid and Symbolic-Numeric Integration era, with affiliations at Heidelberg University [2] and KU Leuven [3]. His contributions focus on the CasADi software framework for nonlinear optimization and optimal control [4], enabling compact, expressive modeling and efficient symbolic–numerical workflows crucial for tackling hybrid problems in this era. CasADi [4] supports automatic differentiation, code generation, and flexible solver interfaces, illustrating the era's emphasis on end-to-end, reproducible toolchains. Through CasADi [4], Diehl [1] helped bridge theory and practice in nonlinear programming, advancing scalable, reusable methods for discrete–continuous modeling and real-time optimization in this era.